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In  a  wide  range  of  decision  tasks  the  decision  maker  faces  a  choice  as 
to  how  much  to  Invest  in  acquiring  decision-relevant  information  before 
making  the  final  decision.  Consumers  invest  in  product-related  information 
before  purchasing  (Jacoby,  1977),  physicians  in  diagnostic  tests  (Elstein, 
Shulman  and  Sprafka,  1978),  employers  in  selection  tests  (Guion,  1976), 
marketers  in  market  surveys  (Chestnut  and  Jacoby,  1982),  and  drilling  com¬ 
panies  in  test  wells  (Raiffa,  1968).  In  each  case,  a  complex  balance  must 
be  struck  between  costs  of  acquiring  information  and  the  benefits  of  improv 
ed  final  decisions. 

In  an  earlier  paper  (Connolly  and  Gilani,  1982)  we  reviewed  the  empiri 
cal  literature  on  human  performance  in  such  information  search  tasks  (see 
also  the  reviews  of  Peterson  and  Beach,  1967;  Hershman  and  Levine,  1970; 
Rapoport  and  Wallsten,  1972;  Slovic,  Fischoff  and  Lichtenstein,  1977;  and 
Einhorn  and  Hogarth,  1981).  The  evidence  suggests  that: 

1.  Information  acquisition  is  somewhat  affected  by  normative  considera¬ 
tions  such  as  Information  quality  and  cost,  but  responses  to  variations  in 
these  factors  are  typically  smaller  than  are  normatively  justified  (e.g. 
Pita,  1968;  Wendt,  1969). 

2.  Normatively  Irrelevant  factors,  such  as  total  information  available, 
may  also  affect  information  acquisition  (e.g.  Levine,  Samet,  and  Brahlek, 
1975;  Fried  and  Peterson,  1969). 

3.  Departures  from  optimal  acquisition  can  be  substantial,  and  appear  in 
decisions  with  real  monetary  stakes  as  well  as  in  play  for  inconsequential 
points  or  poker  chips  (e.g.  Kleiter  and  Wimmer,  1974;  Pitz  and  Barrett, 
1969). 

4.  Learning  of  optimal  acquisition  strategies  is  slow  or  nonexistent  in 
repeated  play  of  laboratory  games  (e.g.  Wallsten,  1968;  Lanzetta  and 
Kaiiareff,  1962). 


5.  Departures  from  optimal  acquisition  may  be  in  the  direction  of  over-  or 
under-purchase  (e.g.  Hershraan  and  Levine,  1970;  Pitz,  1969). 

6.  There  are  substantial  differences  between  individuals  in  their  informa¬ 
tion  acquisition  (e.g.  Pruitt,  1961;  Levine  et  al.,  1975.  See  also 
MacCrimmon  and  Taylor,  1976,  for  a  detailed  review). 

Connolly  and  Gllani  (1982)  also  noted  that  much  of  the  experimental 

work  to  date  has  relied  on  Bayesian  models,  as  in  the  familiar  experimental 

procedure  in  which  subjects  purchase  chips  drawn  from  a  bookbag  before 

betting  on  the  contents  of  the  bag  being  sampled  (Edwards,  1965).  They 

proposed  an  alternative  model  for  the  regression  (or  continuous-variable) 

case,  in  which  the  decision  maker  purchases,  over  a  series  of  trials,  one  or 

more  cues,  X^,  each  of  which  is  correlated  with  a  true  score,  Ye,  which 

the  decision  maker  attempts  to  estimate  or  predict.  Prediction  errors  are 

penalized  in  proportion  to  the  square  of  the  difference  between  the 

subject's  estimate,  Y  ,  and  the  true  score,  Ye>  on  each  trial. 

Specification  of  the  cost,  c^.,  and  validity,  p^  ,  of  each  cue  X^,  and  of 

J  xiye 

2 

the  constant  d  in  the  prediction-error  penalty  function  P^  ■  d(Ye~Yg) 
implies  an  optimal  strategy  for  purchasing  and  using  cues.  In  general,  for 
given  values  of  cue  validity  and  cost,  increasing  the  penalty  for  errors 
Increases  the  number,  N  t,  of  cues  that  will  minimize  the  subject's 
expected  total  cost  (cue  cost  plus  error  penalty). 

Connolly  and  Gilani  reported  the  results  of  two  experiments  using  a 
task  based  on  this  model.  (The  details  of  their  procedure  are  given  below.) 
Limiting  themselves  to  cue  sets  of  equal  cost  and  moderate  validity  (p 

Ve 

■  .85),  they  found: 

1.  That  subjects  tended  to  overpurchase  in  tasks  for  which  optimal  pur- 
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chase  was  low  (N  _  -  1),  and  to  underpurchase  In  tasks  for  which  optimal 
opc 

purchase  was  larger  (N  *3). 

2.  That  subjects  tended  to  purchase  more  heavily  when  eight  cues  were 
available  than  when  offered  a  maximum  of  four  cues. 

3.  That  purchasing  was  heavier  when  the  game  was  presented  In  a  gain-maxi 
mixing,  rather  than  a  loss-minimizing,  format,  though  the  change  of  format 
had  no  effect  in  optimal  strategy. 

4.  That  subjects  in  three  of  the  four  experimental  conditions  improved 
their  purchasing  strategies  with  experience,  though  the  improvement  was 
modest,  and  departures  from  optimal  purchasing  were  still  substantial  in 
late  trials. 

5.  That,  despite  the  statistical  equivalence  of  the  cue-sets  offered, 
subjects  formed  strong  beliefs  as  to  the  differential  validity  of  the  cues, 
and  purchased  in  accordance  with  these  (erroneous)  beliefs. 

In  short,  Connolly  and  Gilani's  data  support  the  earlier  literature  in 
suggesting  that  human  skills  in  balancing  the  costs  and  benefits  of  informa 
tion  acquisition  may  be  seriously  deficient,  even  in  a  laboratory  task  in 
which  the  balance  is  made  highly  salient  and  extended  opportunities  for 
learning  are  provided.  If  these  findings  generalize  to  the  range  of  real- 
world  information-acquisition  tasks  noted  earlier,  they  imply  significant 
non-optimalities  may  be  found  in  such  tasks.  The  Connolly  and  Gilani  find¬ 
ings  were,  however,  restricted  to  tasks  in  which  cues  were  of  equal  cost, 
and  of  equal  and  moderate  validity. 

A  straightforward  extension  of  Connolly  and  Gilani’s  model  shows  that, 
with  other  task  parameters  fixed,  optimal  purchase  strategy,  N  t,  a 
single-peaked  function  of  cue  validity.  Large  purchases  are  normatively 
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justified.  In  general,  only  when  cues  are  moderately  valid.  Low-validity 
cues  do  not  yield  enough  reduction  in  decision  error  to  justify  their  cost; 
and  highly  valid  cues  provide  so  much  information  that  few  are  needed.  An 
illustrative  family  of  optimal  purchase  curves  is  shown  in  Figure  1  for  a 
range  of  values  of  cue  validity  and  cost.  (Note  that  the  curves  are  slight¬ 
ly  displaced  vertically  for  clarity.)  The  analogous  relationship  for  the 
Bayesian  model  is  developed  by  Edwards  (1965).  Snapper  and  Peterson  (1971) 
found,  in  a  task  based  on  Edward's  model,  that  subjects'  actual  purchasing 
behavior  departed  substantially  from  optimum.  Subjects  over-purchased  when 
dlagnosticity  was  low,  under-purchased  at  intermediate  levels  of  diagnos- 
ticity,  and  approximated  optimal  purchasing  only  for  highly  diagnostic  in¬ 
formation.  It  appears,  then,  that  the  nonlinear  relationship  between  infor¬ 
mation  quality  and  optimal  purchase  may  not  be  intuitively  obvious  to  naive 
subjects. 

(Figure  1  about  here) 

The  three  experiments  reported  here  extend  Connolly  and  Gilani's  find¬ 
ings  to  situations  in  which  cue  validity  or  cost  varies.  The  first  examines 
purchasing  behavior  over  a  range  of  cue  validities,  with  cue  cost  and  valid¬ 
ity  equal  within  each  set.  In  the  second  experiment,  cues  offered  were  of 
equal  validity  but  unequal  cost.  In  the  last  experiment,  cues  were  of  equal 
cost  but  unequal  validity.  The  broad  intent  was  to  extend  the  original 
findings  to  approximate  more  closely  the  situation  of  real-world  information 
purchasers  who,  we  assume,  must  generally  select  from  cue  sets  differing  in 
both  validity  and  cost. 
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General  Procedure 

As  In  Connolly  and  Gilanl  (1982),  subjects  were  instructed  that  they 
were  to  act  as  analysts  for  an  imaginary  company.  Game  Predictions  Incorpo¬ 
rated  (GPI),  that  sold  predictions  of  football  game  results  to  its  clients. 
The  predictions  were  based  on  the  assessments  of  “football  experts"  on  re¬ 
tainer  to  GPI.  Each  expert  would,  for  a  fee,  provide  an  assessment  of  the 
likely  pointspread  on  any  game.  Subjects  were  also  told  that  GPI  paid  re¬ 
bates  to  its  clients  when  its  predictions  differed  from  actual  game  results. 
Fees  paid  to  experts  represented  the  information  charge,  c^ ,  in  the  model, 
while  client  rebates  represented  the  penalty  charge,  P^. 

In  a  change  from  the  Connolly  and  Gilanl  (1982)  procedure,  the  experi¬ 
ment  was  conducted  in  an  interactive  mode  on  a  micro-computer.  Instructions 
were  presented  on  the  screen  with  an  opportunity  for  the  subject  to  review 
any  part.  An  experimenter  was  on  hand  to  answer  questions,  and  a  hard-copy 
summary  of  the  key  points  and  the  rebate  chart  was  provided.  The  subject 
first  read  through  the  detailed  task  instructions  explaining  the  activities 
of  GPI,  the  task,  and  the  pointspread  metric  used  both  for  expressing  expert 
assessments  and  for  recording  predicted  and  actual  game  results.  Both 
screen  and  hard-copy  instructions  emphasized  the  goal  of  minimizing  total 
cost  by  balancing  expert  fees  and  rebate  costs.  Finally,  three  sample  dis¬ 
plays  of  the  format  for  expert  preditions  and  game  results  were  shown  to 
familiarize  the  subject  with  the  screen  layout. 

For  each  of  the  experimental  conditions  described  below,  subjects  first 
played,  for  practice,  30  games  described  as  having  been  drawn  from  results 
the  previous  season.  For  these  games,  all  experts  provide  assessments  free 
of  charge.  After  all  assessments  appeared,  the  subject  was  asked  to  enter  a 
prediction  of  how  the  game  would  turn  out.  (S)he  was  then  shown  the  actual 
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game  result,  the  prediction  error  (if  any),  and  the  penalty  (rebate  charge) 
that  would  have  been  charged.  These  practice  games  were  intended  to  allow 
the  subject  to  become  familiar  with  the  experimental  procedure,  and  to 
assess  the  value  of  each  expert  in  relationship  to  actual  game  results  and 
rebate  charges. 

The  second  phase  of  the  experiment  involved  prediction  of  30  "real" 
games,  for  which  expert  fees  and  rebate  charges  were  recorded.  For  each 
game,  the  subject  first  entered  the  total  number  of  experts  (s)he  wished  to 
purchase  on  that  game,  and  the  identifying  numbers  of  those  experts.  Any 
assessments  bought  were  displayed  on  the  screen.  The  subject  then  entered  a 
prediction  of  the  game  result.  The  actual  game  result  was  then  shown,  the 
error  and  rebate  charge  computed,  and  a  summary  of  information  cost  and 
rebate,  both  for  that  game  and  a  running  total  for  all  games  played,  were 
then  displayed.  After  completing  these  30  games,  the  subject  completed  a 
brief  post-experimental  questionnaire  presented  on  the  screen,  and  was  then 
debriefed  and  released  by  the  experimenter. 

The  subjects  were  engineering  students  at  Georgia  Institute  of 
Technology,  satisfying  a  course  requirement  by  laboratory  participation. 

They  were  run  individually,  with  each  experimental  session  lasting  approxi¬ 
mately  45  minutes. 

Task  Parameters 

Actual  game  results  Yg  were  drawn  from  a  Normal  (0,225)  distribution, 
so  that  about  2/3  of  all  games  were  decided  by  pointspreads  of  15  points  or 
less.  Expert  assessments  were  generated  by  adding  to  Yg  an  error  term  nor¬ 
mally  distributed  with  zero  mean  and  variance  selected  to  yield  the  desired 
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cue  validity  (see  below),  with  the  sum  scaled  so  as  to  equate  variance  of 
assessments  and  true  scores. 

Experiment  1 

The  first  experiment  had  two  primary  purposes:  to  examine  the  compara¬ 
bility  of  information-purchase  behavior  between  the  present  computer-inter¬ 
active  format  and  the  manual  procedure  used  by  Connolly  and  Gilani;  and  to 
explore  the  generalizability  of  their  results  across  a  range  of  values  of 
cue  validity.  Three  cue-validity  conditions  were  selected:  .75,  .85  and 
.95.  Correlations  smaller  than  about  .70  are  not  readily  detected  by 
subjects  when  the  pairs  of  numbers  are  presented  sequentially  (Jennings, 
Amabile  and  Ross,  1982).  Correlations  of  .95  are  readily  detected.  The 
intermediate  value  of  .85  replicates  the  value  used  in  the  earlier  studies. 

In  this  experiment,  cue  validity  was  fully  crossed  with  a  second  fac¬ 
tor,  Nopt >  the  optimal  number  of  cues  to  purchase  on  each  trial.  Cue  cost, 
c^j ,  was  kept  constant  at  $10  per  game,  and  the  penalty  cost  constant,  d, 
was  set  to  produce  optimal  purchase  conditions  of  one,  two  or  three  cues  per 
game.  Each  subject  participated  In  only  one  condition  of  cue  validity  and 
.  Six  subjects  were  randomly  assigned  into  each  of  the  nine  conditions, 
for  a  total  of  54  subjects. 

Results 

The  overall  pattern  of  cue  purchasing  is  shown  in  Figure  2.  As  in 

Connolly  and  Gilani  (1982),  subjects  tended  to  overpurchase  in  the  N  t=l 

conditions,  and  to  underpurchase  in  the  N0pt”3  conditions.  More  cues  were 

purchased  in  the  moderate  validity  condition  than  in  the  high  validity 

condition  for  all  three  values  of  N  .  ,  though  this  pattern  of  increasing 

opt 

purchase  with  declining  validity  does  not  extend  to  the  lowest  validity 
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conditions 


(Figure  2  about  here) 

To  examine  the  possibility  of  learning  over  the  30  experimental  games, 
purchasing  behavior  was  computed  separately  for  the  first  and  last  15  trials 
in  each  condition,  allowing  a  3  x  3  x  2  (Repeated  Measure)  ANOVA.  This 
analysis  shows  a  significant  effect  both  for  validity  condition  (F  =  5.01;  p 
<  .01)  and  optimality  condition  (F  =  13.82;  p  <  .001),  with  no  significant 
interaction  (F  **  1.81;  ns).  Neither  validity  nor  optimality  showed  a  sig¬ 
nificant  main  effect  on  learning,  though  there  was  a  significant  interaction 
effect  (F  *  3.14;  p  <  .03).  Purchasing  was  closer  to  optimal  in  later 
trials  for  all  three  high-validity  conditions,  and  for  the  Nopt=3  conditions 
of  the  moderate  and  low-validity  groups. 

The  post-experimental  questionnaire  asked  the  subjects  to  rate  the 
accuracy  of  each  expert  on  a  five-point  scale  anchored  at  1  ("Very  innacu- 
rate")  and  5  ("Very  accurate").  The  four  cues  offered  each  subject  were, 
within  sampling  variability,  of  equal  validity.  They  were,  however,  seen  by 
the  subjects  as  of  widely  different  accuracy:  the  mean  difference  between 
highest  and  lowest  ratings  was  2.1  in  the  .75  validity  condition,  2.0  in  the 
.85  validity  condition,  and  2.1  in  the  .95  validity  condition.  These  rat¬ 
ings  were  also  strongly  associated  with  actual  purchasing  behavior.  The 
correlations  of  ratings  and  frequency  of  buying  of  each  expert  showed  a  mean 
of  .58  for  the  low-validity  cues,  .38  for  the  moderate-validity  cues,  and 
.48  for  the  high-validity  cues  (all  significantly  different  from  zero;  p  < 
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.01).  Mean  ratings  were  not  significantly  different  across  validity  condi¬ 
tions  (F  -  .45;  ns). 

Overall  these  findings  confirm  and  extend  those  reported  by  Connolly 
and  Gilani  (1982).  The  results  reported  here  for  the  .85  validity  and  con¬ 
dition  are  strikingly  similar  to  those  in  the  earlier  study,  suggesting  that 
they  are  robust  to  the  shift  from  the  earlier  manual  procedure  to  the  pre¬ 
sent  computer-interactive  format.  The  pattern  of  overpurchase  for  ^=1 
tasks,  and  underpurchase  for  No{)t=3  tasks  appears  robust  across  variations 
in  cue  validity  from  highly  valid  cues  (.95)  to  levels  at  which  subjects  may 
have  had  difficulty  in  detecting  any  reliable  relationship  between  cue  and 
true  score  (.75).  When  cues  are  highly  valid,  fewer  were  purchased  overall, 
and  modest  learning  was  seen  in  comparing  the  first  and  last  half  of  the 
experimental  trials.  There  is  no  clear  pattern  relating  purchase  of  the 
least  valid  cues  to  optimality  condition. 

Experiment  2 

In  this  experiment,  subjects  were  offered  four  equally  valid  cues,  of 
validity  .75,  .85,  or  .95,  depending  on  experimental  condition.  Two  cues 
were  priced  at  $10  per  game,  the  other  two  at  $20  per  game.  Within  each 
validity  condition,  half  the  subjects  were  offered  Experts  1  and  3  at  $20 
each,  the  other  half  were  offered  Experts  2  and  4  at  this  price.  The  game 
was  otherwise  identical  to  tht  used  in  Experiment  1,  with  penalty  function 
set  so  as  to  make  a  purchase  of  one  $10  cue  per  game  optimal.  Six  subjects 
participated  in  each  cue-validity  condition,  for  a  total  of  18  subjects. 


Results 


Overall  cue  purchasing  behavior  Is  shown  In  Figure  3.  As  In  Experiment 
1,  N0pt=l,  subjects  generally  overpurchased,  though  less  when  the  cues  were 
highly  valid  than  when  validity  was  low.  (The  effect  of  cue  validity  is 
here  not  statistically  significant:  F  *  2.73;  p  <  .11).  The  most  striking 
finding  here,  however,  is  the  extent  to  which  the  more  expensive  (though 
no  more  valid)  cues  were  purchased.  In  the  low  cue-validity  condition,  29% 
of  all  cues  purchased  were  at  the  $20  price;  in  the  moderate-validity  condi¬ 
tion,  21%;  and  in  the  high-validity  condition,  39%  (no  significant  differ¬ 
ence  across  validity  conditions).  Overall,  29%  of  all  cues  purchased  were 
those  bearing  the  higher  price. 

(Figure  3  about  here) 

As  in  the  earlier  experiments,  these  subjects  report  sizable  differ¬ 
ences  in  perceived  accuracy  of  the  experts  they  were  offered.  Only  two  of 
the  18  subjects  rated  all  four  experts  as  equally  accurate  on  the  post-ex¬ 
perimental  questionnaire  scales.  The  mean  difference  between  highest  and 
lowest  rating  was  2.2  scale  points.  However,  unlike  the  previous  studies, 
these  ratings  were  not  significantly  associated  with  buying  behavior;  the 
correlation  of  rating  and  frequency  of  buying  each  expert  was  .10  in  the 
low-validity  condition,  .15  in  the  moderate-validity  condition,  and  .22  in 
the  high-validity  condition.  Mean  ratings  showed  no  significant  difference 
across  validity  conditions  (means  2.6,  2.2,  and  2.6;  n.s.),  nor  across  cue- 
cost  (mean  rating  for  $10  cues:  2.4;  for  $20  cues,  2.5;  n.s.).  It  appears, 
then,  that  the  combination  of  unequal  cost  and  equal  validity  of  cues  left 
the  subjects  somewhat  confused.  They  report  sizable  differences  in  cue 
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validity,  but  these  assessments  appear  unrelated  to  actual  buying  behavior. 
The  most  striking  finding,  however,  is  that,  offered  a  choice  between  equal¬ 
ly  valid  cues,  some  of  which  cost  twice  as  much  as  the  others,  some  one  In 
three  of  all  purchases  were  of  the  more  expensive  cues. 


Experiment  3 

In  this  experiment  subjects  were  offered  four  equally  costly  cues  (at 
$10  each  per  game)  of  unequal  validity.  In  one  condition,  two  of  the  cues 
were  of  high  validity  (.95),  two  were  of  low  validity  (.75).  In  a  second 
condition,  two  cues  were  of  high  validity,  two  of  moderate  validity  (.85) 
and  in  the  third  condition,  two  were  of  moderate,  two  of  low,  validity. 
Within  each  condition,  half  the  subjects  were  offered  the  more  valid  cues  as 
Experts  1  and  3,  while  the  other  half  were  offered  these  cues  as  Experts  2 
and  4.  The  game  was  otherwise  identical  to  that  used  in  Exeriment  1,  with 
penalty  function  set  so  as  to  make  the  purchase  of  one  of  the  more  valid 
cues  per  game  optimal.  Six  subjects  participated  in  each  condition,  for  a 
total  of  18  subjects. 


Results 

Overall  cue  purchasing  behavior  is  shown  in  Figure  4.  Again  parallel¬ 
ing  the  N0pt=l  condition  of  Experiment  1,  subjects  generally  overpurchased 
in  all  three  validity  conditions,  with  no  significant  difference  in  total 
purchasing  across  cue-validity  conditions.  The  most  striking  result,  how¬ 
ever,  is  the  extent  to  which  subjects  purchased  the  less  valid  cues  they 
were  offered.  Overall,  some  28%  of  all  cues  purchased  were  from  the  less 
valid  pair  in  each  mixture.  Purchasing  appears  somewhat  closer  to  optimal 
in  condition  1  (cues  of  .95  and  .75  validity),  where  only  11.4%  of  purchases 
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were  of  the  less  valid  cue,  than  in  conditions  2  and  3,  where  the  less  valid 
cues  accounted  for  35. 72  and  37.62  respectively  of  all  purchases.  The  dif¬ 
ference  across  validity  conditions  fails  to  reach  statistical  significance, 
however  (F  =  1.74;  ns). 


(Figure  4  about  here. 

As  in  earlier  experiments,  subjects  perceived  substantial  differences 
in  the  validity  of  the  cues  offered.  On  the  post-experimental 
questionnaire,  only  two  subjects  rated  the  four  experts  equally  accurate, 
and  the  mean  difference  between  highest  and  lowest  accuracy  rating  was  2.1 
(on  the  5-point  scale).  These  ratings  reflect,  to  some  extent,  the  actual 
differences  in  cue  validity.  In  condition  1  (validity  .95  and  .75),  the 
mean  rating  for  the  more  valid  cues  was  3.5,  against  1.9  for  the  less  valid 
cues  (t  =  5.27;  p  <.002).  In  condition  2  (validity  .95  and  .85),  the  re¬ 
spective  mean  ratings  were  3.2  and  2.6  (t  =  1.34;  ns),  and  in  condition  3 
(validity  .85  and  .75),  the  mean  ratings  were  3.6  and  2.7  (t  =  1.54; 
p  <  .10).  These  assessments  are  also  strongly  related  to  actual  buying 
behavior.  The  correlation  between  accuracy  ratings  and  actual  purchase 
frequency  is  .77  for  condition  1,  .64  for  condition  2,  and  .81  for  condition 
3  (all  significantly  different  from  zero:  p<  .001). 

There  is  evidence,  then,  that  subjects  offered  cues  of  differential 
validity  were  able  to  detect  the  difference,  and  to  shape  their  buying  stra¬ 
tegies  accordingly.  This  insight  into  the  task  conditions,  however,  was 
insufficient  to  produce  a  close  approximation  to  optimal  strategy:  some  one 
of  every  three  cues  purchased  was  less  valid  than  another  cue  offered  at  the 
same  time;  and,  as  in  other  N 


=1  conditions,  the  overall  pattern  was  of 
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substantial  overpurchase. 

Discussion 

-  ^  The  broad  question  addressed  by  this  research  Is:  How  good  are  humans 
at  balancing  the  costs  and  benefits  of  their  information  acquisition?  Do 
they  buy  those,  and  only  those,  sources  of  information  whose  acquisition 
cost  is  outweighed  by  the  improvement  in  decision  quality  that  their  use 
makes  possible?  The  evidence  reported  here,  together  with  that  reviewed 
earlier,  suggests  that  the  answer  is  not  encouraging.  Specifically,  the 
present  findings  extend  those  noted  earlier  in  suggesting: 

1.  That  the  pattern  of  overpurchase  for  low-consequence  decisions,  and 
underpurchase  for  high-consequence  decisions,  is  robust  to  variation  in 
overall  cue  validity,  as  well  as  to  procedural  modifications  such  as  manual 
versus  computer-interactive  transactions  (Experiment  1). 

2.  That  overpurchase  is  frequently  coupled  with  mispurchase  (Experiments  2 
^a^<f  3).  That  is,  subjects,  in  addition  to  buying  overall  more  information 

than  was  normatively  justified,  frequently  bought  expensive  cues  when  cheap, 
equally-valid  ones  were  available  (Experiment  2),  or  low-validity  cues  when 
higher-validity,  equally-costly  cues  were  available  (Experiment  3). 

3.  That  subjects  perceive  equally-valid  cues  as  of  differential  validity 
(Experiments  l-^nd-2),  and  are  able  to  detect  real  validity  differences 
between  cues  reliably  only  when  the  differences  are  large  (Experiment  3). 
Purchase  behavior  is  generally  shaped  by  these  perceptions  of  validity, 
whether  well-founded  or  not,  though  the  relationship  disappears  when  equal¬ 
ly-valid  cues  are  offered  at  different  costs  (Experiment  2). — 

4.  That,  although  modest  learning  was  seen  in  some  cases,  these  departures 
from  optimality  persisted  to  the  end  of  the  experimental  period. 
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It  does  not  appear  that  these  results  are  adequately  accounted  for 
merely  as  manifestations  of  the  "flat  maximum"  problem  (von  Winterfeldt 
and  Edwards,  1982).  Several  investigators  (e.g.  Wendt,  1969;  Rapoport 
and  Wallsten,  1972:  169)  have  noted  that  information-search  models  tend 

to  have  total  cost  curves  with  rather  flat  maxima,  in  the  sense  that 
moderate  deviations  from  optimal  purchase  do  not  greatly  increase  costs 
in  the  general  case.  The  present  model  shares  this  characteristic  (see 
Connolly  and  Gilani,  1982:  335-336).  However,  the  parameter  values 

chosen  for  the  present  experiments  often  led  to  substantial  penalties  for 
purchase  errors.  Table  1,  for  example,  shows  the  percentage  increase  in 
expected  total  cost  resulting  from  over-  or  under-purchasing  by  one  cue 
in  Experiment  1  (assuming  optimal  use  of  information  purchased) . 

(Table  1  about  here) 

Mispurchase  in  cells  where  cues  are  highly  valid  and  optimal  purchase 
small  is  sharply  penalized.  Similarly,  in  Experiment  2,  subjects  buying 
a  single  $20  cue  instead  of  the  $10  cue  could  expect  to  increase  their 
total  costs  by  45.1%  in  the  .75  validity  condition,  by  60.2%  in  the  .85 
validity  condition,  and  by  83.0%  in  the  .95  validity  condition.  In 
Experiment  3,  purchase  of  the  less  valid  cue  of  those  offered  increased 
expected  total  costs  by  33.3%  in  the  ML  condition,  by  37.8 %  in  the  HM 
condition,  and  by  83.8%  in  the  HL  condition.  (These  figures  underestimate 
actual  penalty,  since  the  subjects  in  Experiments  2  and  3  bought  too  many, 
as  well  as  wrongly  selected,  cues).  In  short,  at  least  some  of  the  non- 
optlmal  purchasing  observed  in  these  experiments  persisted  in  the  face  of 
substantial  cost  penalties. 

A  second  possibility  is  that  the  subjects  were  responding  to  some 
loss  function  other  than  the  squared-error  function  used  in  deriving 


optimal  strategies  (c.f.  Peterson  and  Miller,  1964;  Brehmer  and 
Kuylenstierna,  1978).  They  might,  for  example,  be  responding  to  error, 
rather  than  penalty,  despite  the  efforts  made  in  the  experimental  instruc¬ 
tions  to  stress  the  shape  of  the  penalty  function  both  graphically  (on  the 
screen  and  with  a  paper  copy  provided)  and  verbally  (emphasizing  that  large 
losses  were  penalized  proportionally  more  than  small  ones).  It  is,  of 
course,  possible  that  the  subjects  maintained  some  variant  loss  function 
despite  these  efforts.  However,  unless  such  variants  were  related  to 
experimental  condition,  it  is  difficult  to  see  how  they  can  account  for 
both  under-  and  over-purchase.  For  example,  minimizing  error  (rather  than 
squared  error)  would  have  led  subjects  toward  consistent  underpurchase  by 
the  standards  of  the  model.  Variant  loss  functions,  then,  while  certainly 
possible,  do  not  provide  any  obvious  and  parsimonious  explanation  of  the 
data. 

This  is,  evidently,  a  difficult  task  for  the  subjects  to  master  within 
a  30-trial  learning  period:  final  levels  of  performance  are  substantially 
short ^of  optimal,  and  modest  learning  may  continue  throughout  the  experi¬ 
mental  period.  Part  of  the  difficulty  may  stem  from  failure  to  make  correct 
assessments  of  the  validities  of  the  several  cues  presented.  This  latter 
component  of  the  task  is  made  more  difficult  by  the  intercorrelation  between 
the  cues.  A  subject  who  has  reduced  his  or  her  error  by  some  amount  by 
purchasing  one  cue  will  achieve  a  smaller  reduction  by  buying  a  second  cue 
of  the  same  validity,  a  third  cue  will  contribute  still  less,  and  so  on. 

The  subject  may  rate  these  later  cues  as  less  "valid"  than  the  first, 
recalling  that  they  were  less  useful,  on  the  average,  in  reducing  error. 
Though  cues  were,  in  this  experiment,  purchased  en  masse  rather  than 
sequentially  within  each  trial,  the  subjects  may  well  have  attended  to  them 
in  some  sequential  way,  making  difficult  the  assessment  of  their  validity 
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and  thus,  in  turn,  the  development  of  an  optimal  strategy. 

Extrapolation  from  these  findings  to  the  real-world  contexts  noted 
earlier  is,  as  always,  problematic.  On  the  one  hand,  student  subjects  work¬ 
ing  for  a  relatively  brief  period  on  an  unfamiliar  task  in  which  they  have 
no  substantial  stake  clearly  do  not  reflect  many  of  the  features  of  their 
real-world  counterparts.  On  the  other  hand,  the  experimental  procedures 
attempted  to  make  as  salient  as  possible  the  required  balancing  of 
information  cost  and  benefit;  information  charges  and  error  penalties  were 
commensurable  and  updated  each  game;  immediate  feedback  was  provided  over  a 
long  run  of  trials;  adequate  opportunity  was  provided  to  sample  and  assess 
each  of  the  cues  in  the  practice  period  before  the  experimental  trials;  and 
the  cognitive  work  required  to  utilize  the  information  acquired  was 
minimized.  In  short,  while  the  findings  are  clearly  subject  to  the  usual 
laboratory  restrictions,  we  believe  that  they  provide  at  least  a  basis  for 
developing  hypotheses  as  to  real  world  information  acquisition  behaviors. 

The  general  hypothesis  suggested  by  this  and  earlier  studies  is  that 
real-world  information  gathering  may  be  seriously  suboptimal.  The  labora¬ 
tory  evidence  provides  little  basis  for  optimism  that  individuals  will 
routinely  be  able  to  select  the  most  cost-effective  mix  of  informational 
inputs  on  which  to  base  their  decisions.  Elstein  (1976),  for  example, 
reports  extensive  purchasing  by  physicians  of  diagnostic  tests  of  dubious 
value  (through  the  cost  effectiveness  issue  is  here  confounded  by  third- 
party  payment  effects).  The  widespread  practice  of  interviewing  job  appli¬ 
cants,  despite  the  expense  and  poor  predictive  validity  of  the  information 
so  gathered  (see  Guion,  1976),  might  similarly  represent  a  wasteful  use  of 
information  gathering  resources. 

The  issue  is  particularly  pressing  to  the  designers  of  management 
information  systems  and  similar  computer-based  systems  intended  to  support 
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judgmental  decisions  such  as  those  considered  here.  The  designer  of  such  a 
system  would  be  ill  advised  to  place  a  great  credence  either  on  the  decision 
maker's  assessments  as  to  which  information  sources  are  of  value,  or  on  his 
or  her  existing  patterns  of  information  use.  On  the  present  evidence,  it 
appears  that  information  users  are  likely  to  be  poor  sources  of  such  design 
guidance  (see  also  Ackoff,  1967).  Instead,  the  designer  who  must  decide 
which  sources  of  costly  information  to  provide  to  the  decision  maker  must 
make  an  independent  assessment  of  the  costs  and  decision  relevance  of 
possible  sources,  drawing  on  whatever  actuarial  information  is  available  to 
guide  the  choice.  The  decision  maker's  preference  or  practice  in  such 
matters  seems  unlikely  to  provide  adequate  guidance. 

The  overall  thrust  of  the  present  study,  together  with  that  of  Connolly 
and  Gilani  (1982),  is  to  reinforce  and  extend  the  earlier  evidence  based  on 
Bayesian  models  for  serious  departures  from  optimality  in  information  pur¬ 
chasing.  There  appears  to  be  a  consistent  bias  towards  moderate  information 
purchasing,  with  the  result  that  too  little  is  acquired  when  decision  stakes 
are  large,  and  too  much  when  decision  stakes  are  low.  This  pattern  appears 
robust  across  a  moderate  range  of  cue  validities.  Information  users  do  not 
appear  as  reliable  judges  of  differential  validity  within  a  cue  set,  nor  do 
they  consistently  buy  from  the  cheapest  of  available,  equal ly-va l id  sources. 
The  size  of  these  departures  from  optimality  in  the  simplified  laboratory 
context  suggests  that  large  and  costly  departures  may  frequently  be  found  in 
comparable  real-world  settings. 
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Figure  1:  Optimal  Information  Purchase  versus  Cue  Validity 
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Figure  2:  Actual  vs.  Optimal  Information  Purchase,  Experiment 
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